Hierarchical foveation based on wavelets

ABSTRACT

Different images, or sub-images, are rendered at different wavelet decoding rates, the more rapidly decoded wavelets forming a focal region about which less detailed images, or sub-images, are formed. In a preferred embodiment, sets of images or sub-images form levels of a hierarchy, and the wavelet encodings of these images and sub-images are decoded at rates associated with each level of the hierarchy. A single image may be partitioned into sub-images, or regions, that form each level of the hierarchy, or independent images may be arranged in a hierarchical manner, and subsequently rendered using a hierarchy of wavelet decoding schemes. The hierarchy can be developed at a remote site, and a hierarchical wavelet encoding can be transmitted to a local site, so that the hierarchical foveation effect can be rendered while the details of the lower levels of the hierarchy are being communicated.

BACKGROUND OF THE INVENTION

1. Field of the Invention

This invention relates to the field of image processing systems, and inparticular to the communication and presentation of wavelet encodedimages.

2. Description of Related Art

There are several ways to process information hierarchically. Waveletsoffer a highly efficient framework for representing and processinghierarchically structured information or processes. In general terms, awavelet representation of a function consists of a coarse overallapproximation together with detail coefficients that influence thefunction at various scales. In image processing, wavelet encodings areused to encode images via a hierarchy of terms.

A variety of alternative wavelet encoding techniques are common in theart. An example wavelet encoding technique that uses straightforwardaverage and difference terms is presented herein for ease ofunderstanding. Two image elements, such as two pixel values a, b, can beencoded as two other values, Q, Z, where Q=(a+b)/2, the average term,and Z=(a−b), the difference term. To decode the values a, b, from theencodings Q, Z: a=Q+Z/2, and b=Q−Z/2. Four pixel values a0, b0, a1, a2can be represented as two pair of encodings Q0, Z0, Q1, Z1, and the twoaverage terms Q0, Q1 can be represented as encodings Q, Y, where Q isthe average of Q0 and Q1, and Y is the difference between Q0 and Q1.

The use of wavelets offer a number of advantages and efficiencies. Ifall four pixel values are equal, the Q term will equal this value, andeach of the difference terms Y, Z0, and Z1 will be zero. Techniques arecommon in the art for very efficient encodings and representations ofzero-laden encodings. Note also that, because the Q term represents theaverage of the pixel values, the Q term can be used in place of the fourpixel values until the difference terms Y, Z0, and Z1 are received andprocessed. That is, as the difference terms in a wavelet encoding arebeing processed, the average terms can be displayed, thereby presentingan image that iteratively becomes more and more detailed.

FIG. 1 illustrates an example wavelet encoding of a sixteen elementblock A 110, each element 111, 112, . . . representing, for example, thevalue of a picture element (pixel) for rendering on a display screen. Anindex block 110′ is shown in FIG. 1 for ease of reference to theindividual elements of block A 110. That is, the upper left element 111of block A 110 is referenced as a₀, the next lower element 112 isreferenced as a₁, and so on.

Blocks B 120 and Z 130 represents the average and difference blockscorresponding to a pair-wise encoding of the individual elements of theblock A 110. That is, cell b₀ 121 of block B 120 represents the averageof elements a₀ 111 and a₁ 112, and cell Z0 131 of block Z 130 representsthe difference of elements a₀ 111 and a₁ 112.

Blocks C 140 and Y 150 represents the average and difference blockscorresponding to a pair-wise encoding of the individual elements of theblock B 120. That is, cell c₀ 141 of block C 140 represents the averageof elements b₀ 121 and b₁ 122, and cell Y0 151 of block Y 150 representsthe difference of elements b₀ 121 and b₁ 122.

In like manner, blocks D 160 and X 170 represent the average anddifference blocks corresponding to block C 140, and blocks E 180 and W190 represent the average and difference blocks corresponding to block D160.

As discussed above, each of the cells of blocks E 180, D 160, C 140, andB 120 represent the average value of the pixel elements of block A 110contained within the respective cell. The single cell of block E 180,for example, has a value of 0.625, which is the average of all the pixelelements of block A. The upper left cell c₀ 141 of block C 140 has avalue of 0, which is the average of all the pixel elements of block A inthe upper left region ((2+0+(−3)+1)/4). Thus, a display of blocks E 180,D 160, C 140, B 120, and A 110, in that order, will present aprogressively more detailed, or finer-resolution, view of the imagerepresented by the pixel elements of block A 110.

Note that the values of the cells of the block D 160 can be determinedfrom the value of block E 180 and W 190, as discussed above, and asillustrated in FIG. 2. That is, d₀=E+W/2, and d₁=E−W/2. In like manner,the value of the cells of block C 140 can be determined from the valueof this determined block D 160 and the difference block X 170; block B120 can be determined from C 140 and Y 150; and block A 110 can bedetermined from B 120 and Z 130. Thus, the communication of blocks E180, W 190, X 170, Y 150, and Z 130 is sufficient to communicate thesixteen element values a₀-a₁₅ of block A 110. Note also that the blocksE 180 and W 190 contain one value each; X 170 contains two values; Y 150contains four values; and Z 130 contains eight values. Thus, thecommunication of blocks E 180, W 190, X 170, Y 150, and Z 130 requirethe communication of a total of sixteen values, the same number ofvalues contained in the original block A 110.

Wavelets are commonly used to communicate images so that increasinglymore detailed views of the image can be displayed while the moredetailed information is being received. This provides the advantage ofproviding a distraction to the viewer while the image is beingdownloaded, as well as to allow the user to terminate the transmissionwhen it is determined that a sufficient amount of detail has beencommunicated.

BRIEF SUMMARY OF THE INVENTION

It is an object of this invention to exploit the progressive-resolutioncharacteristics of wavelets to provide a rendering of an image, orimages, that takes advantage of the human perceptive system.

As is known in the field of art, a human's visual field is typicallydrawn to a focal point, or focal area, within a scene. As the radialdistance from that focal area increases, the human perceives less andless detail. This phenomenon is termed foveation. Artists purposely adda detail or feature to define such a focal point, to either bring theuser's attention specifically to the feature, or to put the scene inproper perspective, or to diminish the distracting effects caused by alack of focus. Commercial artists take advantage of this phenomenon inadvertising to draw the viewer's attention to the item being advertized.Internet advertising is becoming increasingly popular, with multipleadvertisers typically vying for a viewer's attention. Copending U.S.patent application “EVOLVING ADVERTISEMENTS VIA AN EVOLUTIONARYALGORITHM”, U.S. Ser. No. 09/277,650, filed Mar. 26, 1999, for J. DavidSchaffer, provides a method for evaluating and evolving an advertisingcampaign based on measures of viewer interest, including the number ofusers who select the advertised page, and so on, and is included byreference herein.

In like manner, computer applications are being developed to assistusers in locating items of potential interest among a multitude ofavailable items. Copending U.S. patent application “DEVICE AND METHODFOR A LATTICE DISPLAY”, U.S. Ser. No. 09/282,321, filed Mar. 31, 1999,for Jacquelyn Martino, Nevenka Dimitrova, and Kaushal Kurapati, presentsa system for displaying multiple images in a hierarchical manner,depending upon a measure of each image's potential interest to theviewer, and is incorporated by reference herein. In this referencedpatent application, images having a high potential interest to the userare presented with more prominence than others, where prominence iseffected via the image's size, location, resolution, brightness, bordertrim, and so on.

The expressed object of this invention, and others are achieved byproviding a method and apparatus for rendering wavelets in ahierarchical manner. Different images, or sub-images, are rendered atdifferent wavelet decoding rates, the more rapidly decoded waveletsforming a focal region about which less detailed images, or sub-images,are formed. In a preferred embodiment, sets of images or sub-images formlevels of a hierarchy, and the wavelet encodings of the images orsub-images are decoded at rates associated with each level of thehierarchy. A single image may be partitioned into sub-images, orregions, that form each level of the hierarchy, or independent imagesmay be arranged in a hierarchical manner, and subsequently renderedusing a hierarchy of wavelet decoding schemes. The hierarchy can bedeveloped at a remote site, and a hierarchical wavelet encoding can betransmitted to a local site, so that the hierarchical foveation effectcan be rendered while the details of the lower levels of the hierarchyare being communicated.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention is explained in further detail, and by way of example,with reference to the accompanying drawings wherein:

FIG. 1 illustrates an example prior art wavelet encoding of a sixteenelement image.

FIG. 2 illustrates an example prior art wavelet decoding of a sixteenelement block.

FIGS. 3A-3F illustrate an example hierarchical foveation based onwavelets in accordance with this invention.

FIG. 4 illustrates example timing diagrams of hierarchical progressivelyfiner resolution renderings at each of four levels of an imagehierarchy.

FIG. 5 illustrates an example wavelet decoding sequence to effect ahierarchical progressively finer resolution renderings in accordancewith this invention.

FIG. 6 illustrates an example alternative wavelet decoding sequence toeffect a hierarchical progressively finer resolution renderings inaccordance with this invention.

FIG. 7 illustrates an example flow diagram for a hierarchical waveletrendering system in accordance with this invention.

FIGS. 8A-8D illustrates an example hierarchical rendering of multipleimages in accordance with this invention.

FIG. 9 illustrates an example block diagram of a hierarchical waveletprocessing system in accordance with this invention.

Throughout the drawings, same reference numerals indicate similar orcorresponding features or functions.

DETAILED DESCRIPTION OF THE INVENTION

FIGS. 3A-3F illustrate an example hierarchical foveation based onwavelets in accordance with this invention. FIG. 3A represents a displayarea 300 for an image (not shown). In accordance with this invention,the display area 300 is partitioned into a number of regions 310-341.The regions 310-341 are arranged in a hierarchical manner. The centerregion 310 is the focal region, or top level of the hierarchy; theregions 321-324 about the center region 310 form the next level of thehierarchy; the regions 331-332 form the next level; and region 341 formsthe lowest level. The dashed lines outlining the regions 310-341 areused to indicate that an image is not yet presented on the display area300.

FIGS. 3B-3F illustrate the display area 300 at subsequent intervals oftime. FIG. 3F representing a final rendering of an image (not shown) inthe display area 300. The discrete cells in each FIG. representing theresolution of the rendering in each region, similar to the partitioningsof blocks A, B, C, D, and E of FIGS. 1 and 2. The smaller the cell, thefiner the resolution.

Illustrated in FIG. 3B, the highest level region 310 is partitioned intosixteen cells. The region is labeled 310A, for ease of reference; eachregion is similarly annotated when a change occurs within the region,compared to the prior FIG. The next level regions 321A-324A eachcomprise a single cell, similar to block E of FIGS. 1 and 2. Asdiscussed above, each cell at this level of abstraction will representthe average value of the lower level cells, or pixels, of the image.Thus, each of the regions 321A-324A, for example, will be rendered as asingle uniform color, if the values of the lowest level cells representthe color of each pixel. Region 310A, on the other hand, will berendered as a patchwork of sixteen regions of color, representing theaverage value of the pixels within each of the sixteen regions. That is,the image segment in region 310A are rendered at a finer resolution thanthe image segments within regions 321A-324A. The remaining regions 331,332, 341 are outlined by dashed lines, indicating that the imagesegments at these regions are not yet rendered.

FIG. 3C illustrates the display area 300 at a later time. Asillustrated, region 310B comprises sixty-four cells; regions 321B-324Bcomprise four cells each; regions 331A, 331B comprise two cells each;and region 341A also comprises two cells. As illustrated by the area ofeach cell, region 310B has a finer resolution than regions 321B-324B,which have finer resolution than regions 331A, 331B, which have finerresolution than region 341A.

In like manner, FIGS. 3D-3F illustrate progressively finer resolutionsfor each of the regions. Note that once the region 310C reaches thefinest resolution, it remains at this resolution through subsequentrenderings, until all regions achieve this finest resolution, at FIG.3F. As will be seen from the details presented further, the timerequired to render the finest resolution at FIG. 3F will be onlyslightly longer than the conventional rendering of a single waveletrepresentation of the entire display area 300.

As illustrated in the sequence 3B-3F, the focal region 310 is renderedat a finer resolution than the surrounding regions as the image isprogressively rendered, until all regions are rendered at the finestresolution, illustrating the finest detail. This is particularlybeneficial if the time duration between renderings is long. As would beevident to one of ordinary skill in the art, each of the illustratedrenderings FIG. 3B-3F can be expected to require progressively longerrendering time durations, because each sequential rendering is likely tocontain progressively more cells to process. Thus, by presenting thefully detailed rendering in region 310C at FIG. 3D, before the fullydetailed rendering of the entire image, at FIG. 3F, allows the viewer tofocus on the focal region 310C sooner than a conventional renderingusing a single wavelet representation of the entire display area 300.

The detailed presentation of the focal region 310C sooner than theentire image, in accordance with this invention, provides for a numberof advantages. Because the human visual system naturally includesfoveation, as discussed above, the presentation of less detail inperipheral regions will be less noticeable, and therefore the perceivedreception speed will be improved. That is, for example, the time of therendering of FIG. 3E may be perceived as the “completion” of thereception of the image, and the filling in of the details in theperipheral regions for FIG. 3F may proceed virtually unnoticed, unlessthe viewer specifically focuses on the peripheral regions. Also, asnoted with regard to prior art sequential renderings, the rendering canbe terminated as soon as the viewer determines that sufficientinformation has been received. This is particularly useful forterminating a download of an image when it is determined that the imageis not an image that the viewer desired. As compared to the prior artsystem, however, because the focal region 310 is presented at finerresolutions more quickly than the entire image, a recognition of whetherthe image is one that the user desired will generally occur morequickly. Other advantages are discussed further below.

FIG. 4, lines 4A-4D illustrate example timing diagrams of hierarchicalprogressively finer resolution renderings at each of four levels of animage hierarchy. For ease of reference, the term sub-image is usedhereinafter to refer to the portions of the overall image that isdisplayed in the aforementioned partitioned regions of the display. Line4A illustrates the resolution of the primary sub-image, the sub-imagethat is rendered in the aforementioned primary, or focal, region, suchas region 310 in FIG. 3. Typically, there will be one primary sub-imageand one primary region that serves as the focal point, although multiplesub-images and primary regions may be defined at the upper level of thehierarchy for other applications of this invention. Line 4B illustratesthe resolution of the secondary sub-images, corresponding to the regionsat the next level of the hierarchy, such as regions 321-324 of FIG. 3.Line 4C illustrates the resolution of the tertiary sub-images, and line4D illustrates the resolution of the background sub-images, at the lastlevel of the hierarchy. The vertical scale of each line 4A-4D representsthe resolution, in terms of the finest feature size that can be renderedat that resolution, consistent with the size of the cells in the regionsof FIG. 3. Each of the illustrated steps is a reduction of feature sizeby half, consistent with the processing of each wavelet difference setW, X, Y and Z of FIG. 2, discussed above. As illustrated, the primarysub-image is processed to provide finer resolutions, at 411, 412, 413, .. . , sooner than the secondary sub-images, at 421, 422, . . . , andsooner than the tertiary sub-images, at 431, and the backgroundsub-images, at 441.

At any point in time 490, a rendering of the resolved sub-images can beeffected. As illustrated in FIG. 4, at time 490, each of thehierarchical levels of sub-images have different resolutions, the finerresolution 491 being available at the top level of the hierarchy, andthe coarser resolution 494 at the lowest level of the hierarchy.

FIG. 5 illustrates an example wavelet decoding sequence to effecthierarchical progressively finer resolution renderings in accordancewith this invention. The horizontal axis represents time. The sets 510,511, . . . illustrate wavelet encodings, corresponding to the examplesgiven in FIGS. 1 and 2. Set 510 corresponds to a wavelet encoding of afirst sub-image I comprising sixteen pixels (one average, E₀, andfifteen difference terms corresponding to sets W, X, Y, and Z); set 520corresponds to a wavelet encoding of a second sub-image J comprisingsixteen pixels; and set 530 corresponds to a wavelet encoding of a thirdsub-image K comprising eight pixels. Sub-image I is representative ofeach sub-image at the top of the hierarchy; sub-image J isrepresentative of each sub-image at the next level of the hierarchy; andsub-image K is representative of each sub-image at the following levelof the hierarchy.

Set 510 is decoded to produce a once-decoded set 511, corresponding tothe above described formation of the D set of averages from the Eaverage and the difference W in FIG. 2. The first two elements of theresultant once-decoded set 511, Id₀ and Id₁, are the average values ofthe two-cell partition of the region assigned to the sub-image I. Theonce-decoded set 511 is decoded to produce a twice decoded set 512. Thefirst four elements Ic₀-Ic₃ of the resultant twice-decoded set 512correspond to the four-cell partition of the region assigned tosub-image I. Subsequent decodings eventually produce the set 514 ofpixel values Ia₀-Ia₁₅ of sub-image I.

Sets 520 and 530 are similarly sequentially decoded to produce sets 524,533 of pixel values of sub-images J and K, respectively. Note that inaccordance with this invention, the rates of decoding of the sub-imagesJ and K are progressively slower than the rate of decoding of theprimary sub-image I. That is, at the time that sub-image I has beentwice-decoded, to produce four average values, sub-image J has only beenonce-decoded, to produce two average values, and sub-image K has not yetbeen decoded, and has a single average value. As illustrated in FIG. 5,the rate of decoding sub-image J is half the rate of decoding sub-imageI, until sub-image I is fully decoded, and the rate of decodingsub-image K is half the rate of decoding sub-image J, until sub-image Jis fully decoded. The particular values of decoding rates, and otherdecoding parameters, are optional. For example, the rate of decoding ofeach level can be equal, but the foveation hierarchy is maintained byinitiating the sequential decoding of each level at a progressivelylater time. Of significance to the foveation aspect of this invention isthat the primary sub-image(s) should be decoded to produce a resolutionthat is finer than the other levels of the hierarchy. Other techniquesmay be applied as well; for example, interesting visual effects may beprovided by dynamically determining which sub-images to decode based ona random, or semi-random process.

Note that, in FIG. 5, the decoding processes uses the “upper” N elementsof the current set to produce the next set of average N elements. Thatis, decoding set 510 to set 511 utilizes the upper two elements IE₀, IW₀to produce the average values Id₀ and Id₁; the remaining fourteenelements remain unchanged from set 510 to set 511. In like manner,determining the four next average values Ic₀-Ic₃ of set 512 onlyrequires the upper four elements of set 511. FIG. 6 illustrates anexample alternative wavelet decoding sequence to effect a hierarchicalprogressively finer resolution renderings in accordance with anotheraspect of this invention. The set 690 includes all of the elements ofthe sets 510, 520, and 530 of FIG. 5. The order of the elements in theset 690, however, is the order in which the elements are to be decodedfor rendering. Illustrated in FIG. 6 is a display area 600 having threeregions 610, 620, 630 for displaying the images I, J, and K,respectively. The first three elements of set 690, IE₀, JE₀, and KD₀,are the single average values associated with each image I, J, and K,and apply to each of the three regions 610, 620, and 630, respectively.The next element IW₀ in the set 690 is the first difference term ofimage I, and is used in conjunction with the average IE₀ to partitionregion 610 into two cells, of average values Id₀ and Id₁, as discussedabove relative to FIG. 5. The next two elements IX₀ and IX₁ are used inconjunction with the average values Id₀ and Id₁ to subsequentlypartition region 610 into four cells of average values Ic₀-Ic₃, asdiscussed above. Note that the next element JW₀ is the value of thefirst difference of the sub-image J, and that it occurs in the set 690before the remaining elements IY₀-IZ₇ associated with the sub-image I.Referring to the time sequence illustrated in FIG. 5, the first decoding521 of the J image occurs before the third decoding 513 of image J, andthus, in accordance with this aspect of the invention, the element JW₀needed to effect the first decoding of the image J occurs in the set 690before the elements IY₀-IY₃ needed to effect the third decoding of theimage I. In like manner, each element in the set 690 occurs in thehierarchical order in which it is needed to effect the hierarchicaldecoding in accordance with this invention.

The hierarchical ordering of the elements from multiple sub-imagesprovides for a number of advantages. As illustrated in FIG. 6, thehierarchically ordered set 690 can be processed sequentially to produceincreasingly detailed sub-images in the display area 600. Thus, if theordered set 690 is being sequentially transmitted from a remotelocation, the rendering to the display area 600 can commence whilesubsequent elements are being transmitted, thereby producing a viewableimage before all the details of the image are received. As noted above,if the hierarchical encoding is a foveation-based hierarchy, the speedof recognition of the overall image, based on the details of the focalarea, will also be faster. Other advantages are presented below.

FIG. 7 illustrates an example flow diagram for a hierarchical waveletrendering system in accordance with this invention. At 710, thesub-image parameters for each sub-image are read. These parametersinclude, for example, the hierarchy level of each sub-image, thelocation within the regions allocated for the hierarchy level forrendering each sub-image, the number of sets of differencescorresponding to each sub-image, and so on. The loop 720-726 reads, at724, the parameters associated with each hierarchy level. Theseparameters include when to start the decoding of sub-images at thislevel of the hierarchy, relative to the commencement of the waveletdecoding process, and the time increment between subsequent decodings.As is common in the art, these “time” parameters are specified in termsof “cycles” through the decoding process. The cycle count, orcycle-time, is initialized to zero, at 728.

The loop 730-769 effects the decoding and rendering process for eachlevel of the sub-image hierarchy. At 735, the next scheduled time toprocess images at each level of the sub-image hierarchy is compared tothe current cycle time. If the times match for a particular hierarchylevel, at 735, the loop 740-750 processes each sub-image within theparticular hierarchy level. If, at 745, the sub-image has not yet beencompletely decoded, it is decoded once to produce the next set ofpartitioned cell-averages for this sub-image, at 750.

After all the sub-images at this hierarchy level are process, the nextscheduled process time for the hierarchy level is updated, at 760. Afterall the hierarchy levels are processed, the cycle time is incrementedand the process is repeated until, at 775, all the sub-images have beendecoded. At any time during the process of FIG. 7, the decoded averagesof each sub-image can be rendered, thereby providing a progressivelyfiner display of the sub-images while the lower level details are beingdetermined, as discussed above.

FIGS. 8A-8D illustrates an example hierarchical rendering of multipleimages. As noted above, copending U.S. patent application “DEVICE ANDMETHOD FOR A LATTICE DISPLAY”, U.S. Ser. No. 09/282,321, filed Mar. 31,1999, for Jacquelyn Martino, Nevenka Dimitrova, and Kaushal Kurapati,presents a system for displaying multiple images in a hierarchicalmanner, by rendering images at the upper levels of the hierarchy moreprominently than images at the lower levels. In one embodiment of thereferenced invention, images are related to one another in a latticenetwork; for example, images of a scene from a television broadcast arerelated to images of other scenes from the same broadcast, images of thesame actor in a different broadcast, images of other broadcasts by thesame director, and so on. When an image is selected as the primaryimage, the prominence of all of the images related to the selected imageis determined based on the relative strength of the relationship of eachimage to the selected image. The strength is determined, for example, bythe minimum number of links connecting the images in the latticenetwork. FIG. 8A illustrates a partitioning of a display area 800 intofive regions 810, 820, 830, 840, and 850, each region respectivelycorresponding to progressively lower levels in a hierarchy. FIG. 8Billustrates a further partitioning of each area 820-850 of FIG. 8B,corresponding to image locations 821-825 in region 820, 831-834 inregion 830, 841-847 in region 840, and 851-857 in region 850. There area total of twenty-four image locations identified in FIG. 8B, includingthe primary image location 811 in region 810. In accordance with thereferenced invention, using this partitioning of twenty-four imagelocations, the twenty-three images most closely related to the primaryimage are allocated the locations 821-857 in the order of theircloseness to the primary image, which is allocated to location 811. Eachimage is allocated a specific location 811-857, and appropriatelycropped or letter-boxed to conform to the aspect ratio of each location811-857. That is, each of the twenty-four images correspond totwenty-four sub-images of an overall image provided to the display area.

FIGS. 8C and 8D illustrates sequential rendering of the multiple images,using the principles discussed above. FIG. 8D illustrates anotheradvantage of the hierarchical encoding process discussed above. If thefinest resolution of the display 800 is as illustrated in regions 810and 820 of FIG. 8D, there is no need to communicate further details forthe images corresponding to these regions. The 256 cells of region 810,for example, represent an 8-times wavelet decoding of the image atlocation 811. Sets of wavelet differences beyond the 8^(th) level ofencoding need not be communicated. The sixty-four cells each imagelocation 821-825 of region 820 represent a 6-times wavelet decoding ofthe images at each location 821-825. Therefore, sets of waveletdifferences beyond the 6^(th) level of encoding need not be communicatedfor these images. Alternatively, the 7^(th) and 8^(th) levels ofencoding for the images of location 821-825 can be communicated afterthe time that FIG. 8D is rendered, and stored in a local buffer. When auser subsequently selects an image from the locations 821-825 as theprimary image for rendering at location 811, for example, theseadditional levels of decoding will be available in the local buffer forthe selected image, allowing for a rapid rendering of the selected imageat the finer resolution of location 811. In like manner, the waveletencodings can be purposely truncated to limit the bandwidth required tocommunicate the multiple images. In such an embodiment, for example, theresolution illustrated in FIG. 8D can be a purposeful final display,with a variety of resolutions, wherein the details of an image are notfully communicated unless the image is one of the images assigned to arelatively high position 710, 720 in the display hierarchy.

FIG. 9 illustrates an example block diagram of a hierarchical waveletprocessing system in accordance with this invention. The waveletprocessing system includes an encoding system 900 and a decoding system950. A display partitioner 910 partitions a display area into aplurality of regions, and provides the parameters associated with thepartitioning 911 to an image partitioner 920. The partition parameters911 include, for example, the location of the each region on thedisplay, the size of each region, the hierarchy level associated witheach region, and the like. The image partitioner 920 partitions an image901 into sub-images 921 that correspond to the display partitionsdefined by the parameters 911, as discussed above. The sub-images 921are provided to a wavelet encoder 930 that creates a wavelet encoding931 for each sub-image 921. Alternatively, multiple images 925 can beprovided to the wavelet encoder 930, and each of the multiple images 925are encoded by the wavelet encoder 930 to correspond to the displaypartitions to correspond to the display parameters 911, as discussedabove. Optionally, the individual wavelet encodings 931 can be organizedfor communication to the decoder 950 in a hierarchical order, asdiscussed above, by a hierarchical sequencer 940 to provide ahierarchical transmission 941 that is de-sequenced by a hierarchicalde-sequencer 960 in the decoder 950.

A wavelet decoder 970 receives wavelets 961 corresponding to the encodedwavelets 941. In a preferred embodiment, the display partitionparameters 911 are implicitly included in communicated wavelets 961, forexample, via a header to each wavelet that specifies where the waveletis to be rendered in the display area. The wavelet decoder 970 decodesthe wavelet encodings 961 using the techniques disclosed above, andprovides the resultant renderings to a display device 980. Optionally,the wavelet decoder 970 receives independent wavelets 975 that arerendered to the display device 980 in dependence upon a local displaypartitioner 990. That is, typically the display partitioning and waveletencoding in accordance with this invention occur at a server processor,and the wavelets corresponding to each partition are communicated to aclient processor. Alternatively, wavelets may be communicated to theclient processor in a conventional manner, and the local displaypartitioner 990 provides the parameters 991 for decoding each waveletusing varying rates of decoding, as discussed above.

The foregoing merely illustrates the principles of the invention. Itwill thus be appreciated that those skilled in the art will be able todevise various arrangements which, although not explicitly described orshown herein, embody the principles of the invention and are thus withinits spirit and scope. For example, in a preferred embodiment, the imagesand regions are organized in a hierarchical fashion. A hierarchicalstructure facilitates algorithmic processing techniques. Other,non-hierarchical, partitionings of the display or of the images canprovide for wavelet encodings with differing orderings or rates ofdecoding in accordance with this invention. The examples provided in thefigures are presented for illustration purposes other systemconfiguration and optimization features will be evident to one ofordinary skill in the art in view of this disclosure, and are includedwithin the scope of the following claims.

I claim:
 1. An encoder system comprising: a display partitioner thatpartitions a display area into a plurality of regions, each regionhaving associated parameters, and a wavelet encoder that encodes aplurality of sub-images to produce a plurality of wavelet encodings independence upon the parameters associated with each region, wherein saidwavelet encoder encodes at a first encoding rate at least one of theplurality of sub-images to produce a first wavelet having a desiredfocal point, and wherein said wavelet encoder encodes at least one otherof the plurality of sub-images at a second encoding rate that differsfrom the first encoding rate to produce a second wavelet having acorresponding at least one other sub-image with respect to said firstwavelet.
 2. The encoding system of claim 1, further including an imagepartitioner that partitions an image into the plurality of sub-imagescorresponding to the plurality of regions.
 3. The encoding system ofclaim 1, wherein the display partitioner partitions the display areacorresponding to a foveation effect.
 4. The encoding system of claim 1,further including a sequencer that organizes elements of each of theplurality of wavelet encodings in dependence upon the parametersassociated with each region.
 5. A decoding system comprising: a waveletdecoder that receives a plurality of wavelet encodings, decodes at leastone wavelet encoding of the plurality of wavelet encodings at a firstdecoding rate to produce a corresponding at least one sub-image having adesired focal point, and decodes at least one other wavelet encoding ofthe plurality of wavelet encodings at a second decoding rate thatdiffers from the first decoding rate to produce a corresponding at leastone other sub-image, and a display device, operably coupled to thewavelet decoder that renders an image comprising the at least onesub-image and the at least one other sub-image.
 6. The decoding systemof claim 5, further comprising a de-sequencer that receives ahierarchical wavelet encoding and produces therefrom the plurality ofwavelet encodings.
 7. The decoding system of claim 5, further comprisinga display partitioner that partitions a display area into a plurality ofpartitions, wherein at least one partition is associated with the firstdecoding rate, and at least one other partition is associated with thesecond decoding rate.
 8. The decoding system of claim 7, wherein thedisplay partitioner partitions the display area to facilitate afoveation effect.
 9. A method of encoding an image comprising:partitioning a display area into a plurality of partitions, encoding aplurality of sub-images hierarchically into a plurality of waveletencodings based on the plurality of partitions to facilitate asubsequent decoding of the plurality of wavelet encodings at a pluralityof decoding rates, wherein a first sub-image having a desired focalpoint of a particular sub-image of the plurality of sub-images isencoded into a first wavelet at a first encoding rate, and a secondsub-image is encoded into a second wavelet at a second encoding ratethat differs from the first encoding rate with respect to the firstwavelet, so that said second wavelet corresponds to the second sub-imageof the plurality of sub-images.
 10. The method of claim 9, furthercomprising partitioning an image into the plurality of sub-images basedon the plurality of partitions.
 11. The method of claim 9, wherein thepartitioning of the display area is based on a foveation effect.
 12. Themethod of claim 9, further comprising arranging elements of each waveletof the plurality of wavelet encodings into a sequence of elements basedon the plurality of decoding rates.
 13. The method of claim 12, furthercomprising truncating the sequence of elements to facilitate anoptimized communication of the plurality of sub-images to a decodingdevice.
 14. The method of claim 9, wherein the partitioning of thedisplay area is hierarchical.
 15. A method of decoding a plurality ofwavelet encodings comprising: decoding at least one wavelet encoding ofthe plurality of wavelet encodings at a first decoding rate to produceat least one first sub-image having a desired focal point, decoding atleast one other wavelet encoding of the plurality of wavelet encodingsat a second decoding rate to produce at least one second sub-image, andrendering an image comprising the at least one first sub-image and theat least one second sub-image, wherein the at least one first sub-imageis displayed before the at least second sub-image.
 16. The method ofclaim 15, further comprising de-sequencing a composite wavelet encodingto produce the plurality of wavelet encodings.
 17. The method of claim16, further comprising: receiving the composite wavelet encoding from aremote site, and rendering the image while the composite waveletencoding is being received.
 18. The method of claim 15, furthercomprising displaying the image on a display device.
 19. The method ofclaim 15, further comprising partitioning a display area into aplurality of partitions, wherein at least one partition of the pluralityof partitions is associated with the first decoding rate and at leastone other partition of the plurality of partitions is associated withthe second decoding rate.
 20. The method of claim 19, wherein thepartitioning of the display area is based on a foveation effect.